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Method for the graphical representation of motion in 
video in still images 



The motion of objects in a video clip is not represented in individual still images. 
We may use motion blurring to simulate a real-world camera in the presence of 
moving objects. 

Motion-blur costs a great deal of rendering time, and it only blurs the objects so 
that their contours are unrecognizable. The goal is, to convey information about 
the movement of objects in a still image. To accomplish this goal, illustrative 
techniques from comics may be adopted to depict past and future motions of 
objects in a single image. 

Representing Motion: 

Speedlines are an important stylistic element in comics, and although they are 
not based on an exact physical model, their use is well known. Other means of 
depicting motion in still images include motion arrows and contour repetitions. 
Examples are shown in Figure 1 and Figure 2. In the latter case, earlier positions 
of an object are drawn with less detail or slightly faded, whereas the current state 
of the object is presented in all detail. 




Figure 1: Example to illustrate the use of speedlines for representing motion in line drawings 




Figure 2: Example to illustrate the combined use of speedlines and contour repetition for 
representation of motion in line drawings* 



Generating Speedlines, Contours, and Arrows 

For an automated generation of speedlines, arrows, and contour repetitions, the 
motion information must be extracted from a given video. After executing a 
motion estimation process, we use the resulting 2D motion vector information in 
order to establish the location and the approximate shape of the moving objects. 
The information of motion, shape and location of objects is then used to establish 
the appropriate visibility and the perspective correctness of the speedlines. We 
have to keep track of the contour of every single object. We then calculate a 
motion path from the given keyframe data. 

Human artists invented these graphic motion representations; their generation 
requires some heuristics about where and how to draw them. In general, 
speedlines and contour lines 

- Are drawn in the opposite direction of the movement; 

- Start at "characteristic" points of the moving object; 

- Embrace the minimum and maximum extent of the object; 

- Are more or less equally sized, shaped, and directed, without intersecting 
each other; 

- Are uniformly, but not too regularly distributed. 

Candidates for starting points of speedlines are extreme points lying on the 
object boundary, which yield the outline of the object. The algorithm divides the 
shape into a number of stripes where each stripe can hold one speedline. If a 
stripe contains no such key point, additional points are generated on the object's 
outline. In addition, the speedlines should not stick to an object: the algorithm 
should draw them with an offset. 

Drawing Lines with Style 

The output of the Tenderer consists of an image which has overlayed the 
speedlines. The lines are drawn using line styles, which, in turn, simulate hand- 
drawn pen-strokes, by the superposition of the drawing path and the chosen line 
style. 

In order to achieve an appealing end result little user interaction is necessary, 
since nearly all speedline parameters can be computed based on the data based 
on the estimated motion information. Therefore, length, width, distribution, 
number, etc. are determined by applying given heuristic rules for artistic 
expression. 

Example 

Figure 3 shows countour repitition applied to a keyframe taken out of a video clip. 
We have chosen to render a quite simple contour repetition result. In Figure 4, the 
motion vectors that have been extracted from the given video frames are 
overlayed on a key frame. In Figure 5, the mask is shown that contains the 
information about location and shape of the moving object. 



Within the video source a scene may be chosen. This can be accomplished 
either by user interaction or by means of an automatic scene cut detection 
machnism. It may also be a combination of both options, i.e., the automatic 
mechanism makes a proposal, which may be adopted or overruled and edited by 
the user. 

In the next step a key frame needs to be selected. The key frame serves as the 
basis for the next stage where motion will be estimated and it is also considered 
as the image into which the graphic motion representation will be finally inserted. 
Again, this step can be performed either fully automatically, completely manual or 
semi-automatically. 

The selected key frame along with the other frames in the selected scene are fed 
into the motion estimator that extracts motion information throughout the 
sequence with respect to the selected key frame. The motion information may 
be in the form of dense motion vector fields and may be determined as described 
in the copending application entitled System For Estimating Optical Flow, filed 
June 14, 2000, which application is hereby incorporated by reference. The 
resulting raw motion information may be separated into camera motion and 
object motion. 

In the next stage, for the graphic motion representation not representing camera 
motion but object motion, the video clip and the motion information will be 
compensated for the extracted camera motion. The result is video clip and 
motion information that contains information about the moving objects. 

The cleaned motion information along with the video frames are then processed 
in order to find the location and the approximate shape of moving objects. The 
detected moving objects will then be presented to the user by means of 
highlighting the pertaining pixels. The user is then able to select the object or the 
objects for which graphic motion representation shall be applied. The user has 
the possibility to overrule the automatic extraction process by means of a manual 
interaction. The result of this operation is the object mask for the moving object. 

After the objects are selected the user needs to specify what style shall be used 
for the graphic motion representation. Choices may be speedlines, contour 
repetition or a combination of both, or other types of graphic motion 
representation. This also includes the adjustment of parameters for the 
generation of the graphic motion representation, such as line type or line density. 
More information on artistic features of graphic motion representation may be 
found in the corresponding literature. 

With the information gathered so far the system will render the key frame along 
with the speedlines into a new still image. This image is displayed to the user for 
approval. If the result is approved, the rendered image is stored and the 
operation is finished. If the result is not approved, the user has the choice to jump 
back to either the selection of the key frame or the selection of the detected 
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objects. Other possibilities to modify the design decisions can be implemented 
also. 
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Figure 6: Flow Diagram of Speedline System 



Claims 

• The combination of estimating dense motion vector fields, camera motion 
and object segmentation for the generation of speedlines. 

• Application of the concept of speedlines as a graphical overlay applied to 
natural video frames in order to represent motion in video for very low 
bandwidth transmission systems; 
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Figure 3: Illustration of the effect of representing motion in video by means of a still image and 

countour repitition* 




Description of the system 

In Figure 6, the flow diagram of the system for the semi-automatic generation of 
graphic motion representation enriched still images is depicted. 

A video source is the input to the system. 
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